Indicator for tool state and communication in multiarm telesurgery

ABSTRACT

Medical and/or robotic devices, systems and methods can provide an indicator associated with each manipulator assembly of a multi-arm telerobotic or telesurgical system. The exemplary indicator comprises a multi-color light emitting diode (LED) mounted to a manipulator moving an associated surgical instrument, allowing the indicator to display any of a wide variety of signals. The invention may provide an additional user interface to facilitate communications between the telesurgical system and/or members of a telesurgical team.

CROSS-REFERENCES TO RELATED APPLICATIONS

This application is a divisional of U.S. application Ser. No. 11/478,416filed Jun. 28, 2006, which claims priority to U.S. provisionalapplication Ser. No. 60/695,611 filed Jun. 30, 2005, each of which isincorporated herein by reference.

STATEMENT AS TO RIGHTS TO INVENTIONS MADE UNDER FEDERALLY SPONSOREDRESEARCH OR DEVELOPMENT

NOT APPLICABLE

REFERENCE TO A “SEQUENCE LISTING,” A TABLE, OR A COMPUTER PROGRAMLISTING APPENDIX SUBMITTED ON A COMPACT DISK

NOT APPLICABLE

BACKGROUND OF THE INVENTION

The present invention is generally related to medical and/or roboticdevices, systems, and methods. In an exemplary embodiment, the inventionprovides an indicator associated with one or more robotic manipulatorassemblies for communication of a state of the manipulator assembly orother component of the robotic system, for identification of one or moreparticular manipulators, or the like. The indicator(s) can provide anadditional user interface between the robotic system and, for example, asurgical assistant, system operator, or the like.

Minimally invasive medical techniques are intended to reduce the amountof extraneous tissue which is damaged during diagnostic or surgicalprocedures, thereby reducing patient recovery time, discomfort, anddeleterious side effects. One effect of minimally invasive surgery, forexample, may be reduced post-operative hospital recovery times. Becausethe average hospital stay for a standard surgery is typicallysignificantly longer than the average stay for an analogous minimallyinvasive surgery, increased use of minimally invasive techniques couldsave millions of dollars in hospital costs each year. While many of thesurgeries performed each year in the United States could potentially beperformed in a minimally invasive manner, only a portion of the currentsurgeries use these advantageous techniques due to limitations inminimally invasive surgical instruments and the additional surgicaltraining involved in mastering them.

Minimally invasive robotic surgical or telesurgical systems have beendeveloped to increase a surgeon's dexterity and avoid some of thelimitations on traditional minimally invasive techniques. Intelesurgery, the surgeon uses some form of remote control, e.g., aservomechanism or the like, to manipulate surgical instrument movements,rather than directly holding and moving the instruments by hand. Intelesurgery systems, the surgeon can be provided with an image of thesurgical site at the surgical workstation. While viewing a two or threedimensional image of the surgical site on a display, the surgeonperforms the surgical procedures on the patient by manipulating mastercontrol devices, which in turn control motion of the servomechanicallyoperated instruments.

The servomechanism used for telesurgery will often accept input from twomaster controllers (one for each of the surgeon's hands) and may includetwo or more robotic arms or manipulators on each of which a surgicalinstrument is mounted. Operative communication between mastercontrollers and associated robotic arm and instrument assemblies istypically achieved through a control system. The control systemtypically includes at least one processor which relays input commandsfrom the master controllers to the associated robotic arm and instrumentassemblies and back from the instrument and arm assemblies to theassociated master controllers in the case of, e.g., force feedback orthe like. One example of a robotic surgical system is the DA VINCI®system available from Intuitive Surgical, Inc. of Mountain View, Calif.

The roles and interaction among the persons making up a telesurgicalteam may differ from those of conventional surgery. As telesurgery isoften performed in an internal surgical site, at least some of thoseworking under the direction of the lead surgeon (or other robotic systemoperator) may not have direct access to or direct visualization of theongoing tissue manipulations. For example, surgical assistants withinthe operating room may remove a first surgical instrument (such as anelectrosurgical scalpel) from a manipulator and replace it with adifferent surgical instrument (such as a needle holder), as a surgeonmay desire the use of different surgical instruments during differentphases of a surgical procedure. Similarly, the assistant may repositiona surgical instrument during a procedure, so that rather thanapproaching the procedure from a first minimally invasive access site,the instrument is advanced toward the internal surgical site from asecond, different access site. More complex robotic systems (and teaminteractions) may also be used. For example, as surgery is oftenperformed with more than two tools, input devices may be provided forone or more additional surgeons, with each additional surgeonrobotically controlling (for example) at least one surgical instrument.

While the new telesurgical systems, devices and methods have provenhighly effective and advantageous, still further improvements would bedesirable. In general, it would be desirable to provide improved roboticand/or surgical devices, systems and methods, particularly forperforming telesurgical procedures. It may also be desirable to provideimproved techniques for communication among the members of atelesurgical team, and for interfacing with the telesurgical apparatusso as to more fully take advantage of the capabilities of telesurgery toprovide enhanced patient outcomes with improved efficiencies. It may beparticularly beneficial to avoid unnecessary interruptions anddistractions of a surgeon or other system operator, and to avoid delaysand/or mistakes in the coordinated activities of a telesurgical team.

BRIEF SUMMARY OF THE INVENTION

The present invention is generally related to medical and/or roboticdevices, systems and methods. An exemplary embodiment of the presentinvention provides a visual indicator associated with each manipulatorassembly (and, hence, each surgical instrument) of a multi-armtelesurgical system. The exemplary indicator comprises a multi-colorlight emitting diode (LED), allowing the indicator to display any of awide variety of signals through combinations of color, blinking, and thelike. By establishing and using an appropriate indicator signallexicography, the indicator can identify, for example:

-   -   whether the manipulator and surgical instrument are following        input movement commands of a lead surgeon;    -   whether movement of the surgical instrument is inhibited (for        example, when the manipulator is in a clutch mode to allow        manual movement);    -   whether the surgical instrument is at, near, or beyond its        intended useful life;    -   whether a second surgeon has control of the surgical instrument;    -   whether a fault is preventing robotic movement of the surgical        instrument; and/or the like.        A surgeon may also optionally initiate a signal from one or more        of the indicators so as to identify an associated surgical        instrument to be removed and replaced, or the like. As the        surgeon will often remain immersed in the ongoing surgical        procedure, and as it may be difficult to determine which        instrument was, for example, previously being controlled by the        surgeon's right hand and is now to be replaced, such indicators        may significantly facilitate communications between the        telesurgical system and/or members of a telesurgical team,        potentially increasing efficiencies and enhancing patient        outcomes. An identifiable signal is output from the selected        indicator to a person near by.

In a first aspect, the invention provides a robotic method comprisingmoving a first end effector of a first manipulator assembly in responseto a first robotic command. The first manipulator assembly has a firstindicator. A second end effector of a second manipulator assembly ismoved in response to a second robotic command. The second manipulatorassembly has a second indicator. A command is transmitted from aprocessor, with the command being selectably directed to the firstindicator and/or the second indicator.

In many embodiments, the end effectors will include the working ends ofsurgical instruments. The person receiving the identifiable signal willoften be a member of a robotic surgical team within an operating room,with the processor acting as a master/slave telesurgical controller. Insuch embodiments, a movement command may be input by a system operator,and the system operator may also select which of the manipulatorassemblies is to output the first signal. This can allow the systemoperator to maintain his or her concentration on movements of the endeffectors shown in a display while communicating to, for example, anassistant regarding a specific surgical instrument. For example, ratherthan telling the assistant to “replace the electrosurgical scalpel onthe arm now wiggling back and forth with a needle driver,” the surgeoncan instead merely request a needle driver and activate the indicator ofthe appropriate manipulator assembly. Optionally, the processor mayselect the appropriate indicator and manipulator assembly, such as whenthe tool mounted on a manipulator reaches or exceeds its intended usefullife.

The indicator will often include a visual indicator, with the firstsignal including a visual pattern such as blinking or the like, anidentifiable color, or some other identifiable visual signal. In anexemplary telesurgical system, the indicators each include a pluralityof light emitting diodes (LED). A variety of alternative indicatorsmight also be employed, including any of the wide variety of lightsources capable of generating different colors, audible indicators forgenerating tones or verbal signals, and the like. Preferably, aplurality of separately identifiable signals can be output from eachindicator, thereby providing a user interface for communication to theassistant or the like regarding an associated manipulator assembly.

Typically, each manipulator assembly will include a manipulator movablysupporting a tool holder, and a tool releasably mounted to the toolholder. The tool will generally include the end effector. The firstidentifiable signal may indicate, for example, a state of the tool,which specific manipulator assembly is operatively associated with aspecific input device, an overall condition or state of all of themanipulator assemblies, an identity of one or more components of theselected manipulator assembly, or the like. Some listing or lexicographyexplaining the meaning of each identifiable signal will often beprovided or known by the person receiving the signal, and that personwill often take action in response to the signal, such as reconfiguringthe manipulator assembly from which the signal is being output. Thereconfiguration may include replacement of a tool of the manipulatorassembly with another tool having a different (or the same) endeffector, changing a mode of the manipulator assembly so as to initiateor inhibit master/slave movement of the end effector, or the like.

It will often be beneficial to include readily identifiable signal typesor groups. For example, when the first signal includes a yellow color,it may communicate a warning that the associated manipulator assemblyremains functional, but needs attention now or in the near future. Asignal including a red color may communicate a sufficient fault alongthe robotic system chain to interrupt end effector movement.

A variety of refinements may be implemented with use of the manipulatorassembly indicator. For example, the system operator who is inputtingmovement commands may benefit from graphical indicia in his or herdisplay that corresponds to the first identifiable signal. Hence, whenthe system indicates that a tool is near the end of its life or the likeby flashing the indicator, the same information may be communicated tothe system operator by simple flashing of an icon on the operator'sdisplay screen, by superimposing a flashing color on the end effector ofthe instrument as shown in the display, or the like. In someembodiments, along with indicating which currently mounted tool or endeffector is to be replaced, the processor may also energize anotherindicator associated with the new tool that is now desired, such as byflashing an LED at an appropriate location on a tool tray, or the like.

In another aspect, the invention provides a surgical robotic methodcomprising inputting a robotic movement command from a system operatorto an input device. A surgical end effector of a manipulator assemblymoves in response to the robotic command. The system operator views themovements of the end effector in a display while inputting the command.An identifiable visual signal is transmitted from a visual indicator ofthe manipulator assembly to a person nearby. In response, the personmanually reconfigures the manipulator assembly.

In yet another aspect, the invention provides a surgical robotic systemcomprising a first manipulator assembly having a first indicator and afirst end effector. The first end effector is movable in response to afirst movement command. The second manipulator assembly has a secondindicator and a second end effector. The second end effector is movablein response to a second movement command. A processor is coupled to thefirst and second indicators, and selectively induces at least one of theindicators to output a first signal identifiable by a person nearby.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic plan view of a portion of an operating theaterillustrating a robotic surgical system in use, including a mastersurgeon console or workstation for inputting a surgical procedure and arobotic manipulator system for robotically moving surgical instrumentshaving surgical end effectors at a surgical site within a patient.

FIG. 2 is a perspective view of an exemplary manipulator system,including positioning linkages or set up joints which allow two patientside robotic manipulators and one endoscope or camera roboticmanipulator to be pre-configured for surgery.

FIG. 3 is a perspective view of an exemplary articulated surgicalinstrument for use in the system of FIG. 1.

FIG. 4 is a perspective from above of an alternative manipulator systemincluding a plurality of positioning linkages, each supporting amanipulator assembly.

FIG. 5 is a perspective view of an exemplary manipulator for use in themanipulator system of FIG. 4.

FIG. 6 is a detailed view of a portion of the manipulator of FIG. 5,showing a tool holder and adjacent indicator for outputting visualsignals.

FIG. 7 shows the indicator of FIG. 6 in isolation.

FIG. 8 is a schematic block diagram of an embodiment of an alternativerobotic system and method for its use by a plurality of systemoperators.

FIG. 9 is a flow chart schematically describing a method forreconfiguring a manipulator assembly in response to an identifiablesignal from an associated indicator.

FIG. 10 is a listing of separately identifiable visual signals which maybe then generated by the indicator of FIG. 6, along with theirassociated meanings.

FIG. 11 illustrates colors which can be generated by the indicator ofFIG. 6, along with the general meanings of those colors.

FIG. 12 is a listing of indicator signal colors and associated generalmeanings.

FIG. 13 is a listing of indicator signals and associated specificmeanings.

FIG. 14 illustrates An exemplary quick reference guide to the meaningsof indicator signals for a telesurgical system.

DETAILED DESCRIPTION OF THE INVENTION

The present invention generally provides improved robotic and/or medicaldevices, systems and methods. Embodiments of the present invention areparticularly well suited for telesurgery, often providing an additionalform of user interface that can enhance communication between a surgicalsystem operator, an assistant, any other members of a telesurgical teamand/or the telesurgical system. Other embodiments of the invention maybe well suited for use in other telerobotic or robotic environments,particularly with robotic systems having a plurality of manipulators.

FIGS. 1-3 illustrate components of a robotic surgical system 1 forperforming minimally invasive robotic surgery. System 1 is similar tothat described in more detail in U.S. Pat. No. 6,246,200, the fulldisclose of which is incorporated herein by reference. A system operatorO (generally a surgeon) performs a minimally invasive surgical procedureon a patient P lying on an operating table T. The system operator Omanipulates one or more input devices or masters 2 at a surgeon'sconsole 3. In response to the surgeon's input commands, a computerprocessor 4 of console 3 directs movement of endoscopic surgicalinstruments or tools 5, effecting servomechanical movement of theinstruments via a robotic patient-side manipulator system 6 (acart-mounted system in this example).

Typically, manipulator system 6 includes at least 3 robotic manipulatorassemblies. Two linkages 7 (mounted at the sides of the cart in thisexample) support and position manipulators 8 with linkages 7 ingenerally supporting a base of the manipulators 8 at a fixed locationduring at least a portion of the surgical procedure. Manipulators 8 movesurgical tools 5 for robotic manipulation of tissues. One additionallinkage 9 (mounted at the center of the cart in this example) supportsand positions manipulator 10 which controls the motion of anendoscope/camera probe 11 to capture an image (preferably stereoscopic)of the internal surgical site. The fixable portion of positioninglinkages 7, 9 of the patient-side system is sometimes referred to hereinas a “set-up arm.”

The image of the internal surgical site is shown to operator O by astereoscopic display 12 in surgeon's console 3. The internal surgicalsite is simultaneously shown to assistant A by an assistance display 14.Assistant A can help prior to and during a surgical procedure. Prior tosurgery, assistant A typically covers at least a portion of the systemwith sterile drapes. Such draping of the system may include attachingsterile adapters, cannulas, and/or the like to manipulators 8, switchingone or more of the manipulators into and out of a manually articulatableclutch mode, and the like. Assistant A also assists in prepositioningmanipulator assemblies 8 and 10 relative to patient P using set-uplinkage arms 7, 9; in swapping tools 5 from one or more of the surgicalmanipulators for alternative surgical tools or instruments 5′; inoperating related non-robotic medical instruments and equipment; inmanually moving a manipulator assembly so that the associated toolaccesses the internal surgical site through a different aperture, andthe like. Some or all of these activities may be facilitated byproviding feedback to assistant A via the indicators described herein,with the indicators often providing feedback to the assistant regardinga state of the manipulator assembly, a state of the tool mountedthereon, and/or a state of the robotic surgical system.

In general terms, the linkages 7, 9 are used primarily during set-up ofpatient-side system 6, and typically remain in a fixed configurationduring at least a portion of a surgical procedure. Manipulators 8, 10each comprise a driven linkage which is actively articulated under thedirection of surgeon's console 3. Although one or more of the joints ofthe set-up arm may optionally be driven and robotically controlled, atleast some of the set-up arm joints may be configured for manualpositioning by assistant A. Each associated combination of a set-upjoint, manipulator and tool is encompassed within the term “manipulatorassembly” as that term is used herein, although some manipulatorassemblies may not include set-up joints, and the manipulator assemblywill not include a tool at all times.

For convenience, a manipulator such as manipulator 8 that is supportinga surgical tool used to manipulate tissues is sometimes referred to as apatient-side manipulator (PSM), while a manipulator 10 which controls animage capture or data acquisition device such as endoscope 11 may bereferred to as an endoscope-camera manipulator (ECM). The manipulatorsmay optionally actuate, maneuver and control a wide variety ofinstruments, tools, image capture devices and the like which are usefulfor surgery.

As can be seen in FIGS. 1 and 2, indicators 20 are disposed on eachmanipulator assembly. In the exemplary embodiments, indicators 20 aredisposed on manipulators 8, 10 near the interface between themanipulators and their mounted tools 5. In alternative embodiments,indicators 20 and may instead be disposed on set-up joints 7, 9, ontools 5, elsewhere on manipulators 8, 10, or the like, with theindicators preferably being sufficiently close to the tools so that asignal generated by a particular indicator can be readily associatedwith a particular tool when the signal is viewed by assistant A. In FIG.1, additional indicators 20 are shown in a rack or other structuresupporting additional tools 5′, with each indicator again beingassociated with a particular tool (or type of tool).

As can be seen in FIG. 1, the system operator O is largely immersed inthe environment of and interaction with workstation 3. The systemoperator sees images presented by display 12 and manipulates inputdevices 2, and in the exemplary embodiment, processor 4 correlates themovement of the end effectors of tools 5 so that the images of the endeffectors follow the movements of the input devices in the hands of thesystem operator O.

It can be advantageous to avoid distracting system operator O during asurgical procedure so as to require the operator to shift his or herattention away from the internal surgical site. Even when assistant A isreplacing a tool 5 with an alternative tool 5′, the system operator maycontinue manipulating tissues with another tool, or may want to continueto view the surgical site to monitor bleeding, and/or the like.Nonetheless, it may be difficult for the system operator to communicateclearly with assistant A regarding which tool 5 is to be replaced by analternate tool. Specifically, as endoscope 11 may be at any convenientorientation, the tool associated with the right hand of system operatorO will often not be disposed to the right of the endoscope, particularlyas viewed by the assistant.

So as to unambiguously identify a tool 5 to be replaced assistant A,system operator O may input a command into workstation 3 (such as bypushing a button on the input device 2, actuating foot peddle, inputtinga voice command, or the like) so that indicator 20 on the manipulatorassembly associated with the specific tool 5 generates a visuallyidentifiable signal that can be viewed by the assistant. A correspondinggraphical indication of the signal may be provided to system operator Oin display 12, such as presentation of an appropriate icon,superimposing of text on the surgical site, the use of image processingto superimpose false colors on the image of the appropriate endeffector, or the like. In response to the signal on indicator 20,assistant A can remove and replace the identified tool 5. Optionally,each alternative tool 5′ may also have an associated indicator 20,allowing the processor to transmit a signal so as to indicate whichalternative tool the system operator O is to be mounted on the roboticsystem.

There are a number of additional uses for indicators 20 in telesurgicalsystem 1. For example, assistant A will often manually position tools 5and endoscope 11 when setting up for a surgical procedure, whenreconfiguring the manipulator system 6 for a different phase of asurgical procedure, when removing and replacing a tool with an alternatetool 5′, and the like. During such manual reconfiguring of themanipulator assembly by assistant A, the manipulator assembly may beplaced in a different mode than is used during master/slave telesurgery,with the manually repositionable mode as sometimes being referred to asa clutch mode. The manipulator assembly may change between the tissuemanipulation mode and the clutch mode in response to an input such aspushing a button on manipulator 8 (or some other component to themanipulator assembly) thereby allowing assistant A to change themanipulator mode. By generating appropriate visually identifiablesignals with indicators 20 whenever the manipulator is in clutch mode,assistant A can avoid errors and increase the efficiency of the surgicalprocedure.

Indicators 20 may also show an associated identifiable signal when, forexample, no tool is attached to the manipulator, when a sterile adaptoror interface between tool 5 and manipulator 8 is not properly attached,if an instrument mounted onto a manipulator is incompatible with thatmanipulator, if the instrument has reached the end of its useful life,and/or the like. For some of these signals, system operator O mayinitiate generation of the signal and may specify the associatedmanipulator assembly from which the signal is to be produced. In others,processor 4 may initiate the signal and/or indicate which manipulatorassembly(s) are to be indicated. For example, in the case of a powerfailure, robotic system 1 may continue to operate using backupbatteries. So as to indicate to the assistant A that a power failure hasoccurred, indicators 20 on all manipulators may blink, optionallyblinking with a yellow light as a warning. As the power in the batteryis drained so that robotic system 1 can no longer continue to providerobotic movement of the tools, all indicators 20 may blink red, therebyindicating a system fault. A wide variety of alternative signals mayalso be provided, some of which are indicated by the exemplarylexicographies of FIGS. 10-14.

FIG. 4 illustrates a perspective view of the articulated surgical toolor instrument 5. Tool 5 has a proximal housing 24 which interfaces witha tool holder of the manipulator, generally providing a quick releasemounting engagement through a sterile adapter or interface. Tool 5includes an elongate shaft 23 supporting an end effector 28 relative toproximal housing 24. Proximal housing 24 includes a drive mechanism thataccepts and transmits drive signals or drive motion between themanipulator 8 and the end effector 28. Proximal housing 24 also containscircuitry that can generate signals for transmission to processor 4 soas to identify a type of the tool, indicate a remaining useful life ofthe tool, and/or the like. The exemplary circuitry may include a memory,such as a Dallas™ part or the like, and exemplary structures and methodsfor transmitting of information between the memory of tool 5 andprocessor 4 may be described in application Ser. No. 10/839,727, filedon May 4, 2004 and entitled “Tool Memory-Based Software Upgrades forRobotic Surgery” the full disclosure of which is incorporated herein byreference. An articulated wrist 29 may provide two degrees of freedom ofmotion between end effector 28 and shaft 23, and the shaft may berotatable relative to proximal housing 24 about the axis of the shaft soas to provide the end effector 28 with the three orientational degreesof freedom within the patient's body. Control over actuation of endeffector 28, such as opening or closing of jaws or the like, mayrepresent an additional degree of freedom, for a total of four distaldegrees of freedom. In some embodiments, an indicator 20 may be mountedon housing 24.

Referring now to FIG. 4, a perspective view from above of an alternativemodular manipulator support assembly 30 may be mounted to a ceiling ofan operating room. The modular manipulator support 30 aligns andsupports a robotic manipulator system relative to a set of desiredsurgical incision sites in a patient's body. Modular manipulator support30 generally includes an orientating platform 36 and a plurality ofconfigurable set-up linkage arms 38, 40, 42, 44 coupleable to theorienting platform. Each arm movably supports an associated manipulator32, 34, which in turn movably supports an associated tool or an imagecapture device. Orienting platform 36 also supports an assistant display104, which may be used for set-up, instrument changes, viewing of theprocedure and the like. The structures and use of any of the componentsof modular manipulator support assembly 30 are analogous to thosedescribed above regarding manipulator system 6, and are more fully anddescribed in copending U.S. patent application Ser. No. 11/043,688,filed on Jan. 24, 2005, and entitled “Modulator Manipulator Support ForRobotic Surgery”, the full disclosure of which is incorporated herein byreference. As generally described above, each manipulator 32, 34 ofmodular manipulator support 30 may include an indicator 20.

Manipulator 32 is shown in more detail in FIG. 5, which also showsindicator 20 near a tool holder 52. The exemplary indicator 20 is shownin more detail in FIG. 6, and in isolation (with some of the internalcomponents being schematically shown by dashed lines) in FIG. 7. Theexemplary indicator 20 generally comprises a clear and/or translucentpolymer body 54 in which a pair of light emitting diode arrays (LED) 56are mounted. LED 56 include a red LED, a blue LED and a green LED. TheLEDs of each array can be independently energized to any of a pluralityof illumination or brightness levels, allowing indicator 20 to generatesignals in a wide variety of colors, as schematically illustrated inFIG. 11. Additionally, the overall level of illumination from indicator20 may be varied, and a pattern may be imposed on the cover signal byblinking and the illumination on and off, alternating between twodifferent colors, alternating between two different illumination levels,simultaneously displaying two different colors, or the like. The speedof blinking may also be controlled or altered in more complex modulationpatterns (with long and short blinks, Morse code, and the like).

As different colors may be combined with different illumination levelsand different modulation patterns in a signal, a very large number ofindependently identifiable signals can be generated, often being morethan three separately identifiable signals, typically being more than 10separately identifiable signals, and optionally of being more than 100separately identifiable signals. Interpretation of the identifiablesignals may be facilitated by a listing correlating signals and theirmeaning, such as that included as FIG. 10. In some embodiments, arelatively simple signal scheme with a more limited number of signalsoutput by indicator 20 may be combined with supplemental text orgraphics output from the assistant display 114 or the like. Hence, theassistant may know that they should look for additional information onthe assistant display in response to one or more signals.

So as to facilitate interpretation of the signals generated byindicators 20, any of a wide variety of signal conventions may beestablished and/or employed. For example, as can be understood withreference to FIG. 11, signals including the color yellow may generallyindicate a warning. Optionally, the manipulator assembly may continue tofunction while its indicator 20 displays such a yellow warning signal,but the assistant may understand that some action may be advisable. Thespecific action or meaning of the warning signal may depend on otheraspects of the signal, such as other colors being interspersed with ablinking yellow illumination pattern, speed of blinking, or the like.Signals including red may indicate a current or imminent fault that isnow or may soon interfere with operation of the manipulator assembly.Green may optionally indicate normal operation is underway, blue mayindicate an instrument or a tool is engaged, and/or that a guided toolchange is underway or has been requested. Purple may indicate that asecond surgeon or system operator is actively controlling themanipulator assembly, or the like. Slow blinking lights may generallyindicate a less urgent state than a rapidly blinking indicator 20. Someof these possible conventions are graphically illustrated in FIG. 11.

Referring now to FIG. 8, a schematic block diagram of a somewhat morecomplex system 60 is provided. Exemplary cooperative telesurgicalsystems which may be modified to take advantage of the additional userinterface provided by indicators 20 is described in more detail in U.S.Pat. No. 6,659,939, the full disclosure of which is incorporated hereinby reference. In the exemplary system of FIG. 8, a first system operatorO1 provides input to a first input device 2. Processor 4 selectivelycouples the movement commands from the first input device 2 to any of aplurality of manipulator assemblies, for example, to manipulator A andmanipulator D. A second system operator O2 inputs movement commands to asecond input device 2 so as to effect movement of a manipulator 32manipulator assembly B.

When the second system operator O2 desires the tool 5 attached tomanipulator assembly B be replaced an alternate tool 5′, for example,tool F, the system operator may verbally instruct assistant A to mounttool F to his manipulator assembly. The indicator 20 of manipulatorassembly B generates a signal in response to an appropriate input by thesecond system operator O2, clearly indicating to the assistant A whichtool from which manipulator assembly is to be replaced.

As noted above, additional indicators may optionally be associated witheach of the alternative tools 5′, thereby foregoing any need for verbalinstruction to the assistant regarding which alternative tool is to bemounted to the manipulator. Toward that end, alternative tools 5′ may beincluded in a tool rack 62 which is coupled to processor 4, often so asto provide communication between the circuitry of alternative tools 5′and the processor. This can allow the processor to read identifyinginformation from the tools, thereby allowing the processor to determinewhich alternative tools or tool types are available and/or appropriateto mount on the manipulator system. An LED or other indicator may beassociated with each alternative tool 5′ included in the tool rack, andprocessor 4 may transmit a signal to the tool rack so as to energize oneor more indicator associated with one or more tools. Hence, when systemoperator O1 indicates to processor 4 a desire to replace a mounted tool5 (for example, tool B on manipulator B) with a different type of tool,a list of available tools and/or tool types may be displayed to thesystem operator. In response to the system operator selecting (forexample, a tool type corresponding to tool F, indicators 20 associatedwith tool B and tool F may be activated, indicating to the assistant Athat these two tools should be exchanged.

Processor 4 will typically include data processing hardware andsoftware, with the software typically comprising machine-readable code.The machine-readable code will embody software programming instructionsto implement some or all of the methods described herein. Whileprocessor 4 is shown as a single block in the simplified schematic ofFIG. 8, the processor may comprise a number of data processing circuits,with at least a portion of the processing optionally being performedadjacent input device 1, a portion being performed adjacent manipulatorB, and the like. Any of a wide variety of centralized or distributeddata processing architectures may be employed. Similarly, theprogramming code may be implemented as a number of separate programs orsubroutines, or may be integrated into a number of other aspects of therobotic systems described herein.

Referring now to FIG. 9, a flowchart 70 schematically illustrates amethod for implementing an embodiment of the present invention. Themethod 70 may begin with a change 72 in state of the manipulator, tool,system, or the like. For example, an assistant may actuate a clutch modebutton on the manipulator, the tool may reach the end of its usefullife, a manipulator fault may be detected, or the like. In otherembodiments, the system operator may initiate method 70 by requesting anew tool 74, or in some other manner indicating which of one or moremanipulator assemblies is to generate a signal to be perceived by anassistant or some other person.

Regardless of whether the signal is initiated by a change in state or asystem operator, the processor will typically transmit an appropriatecommand to one or more of the manipulator assemblies 76, and, inresponse, the indicator on that manipulator assembly will display avisual signal 78. Optionally, corresponding graphics may be shown in asystem operator display 80, thereby allowing the system operator tomaintain his or her concentration on the internal surgical site.

In response to the visual signal, the other person, such as anassistant, may optionally reconfigure the manipulator assembly 82. Forexample, the assistant may remove and replace a tool (with its endeffector) or may manually reposition the manipulator linkage so as tomove the end effector into a desired position. In some embodiments, theassistant may merely monitor the manipulator assembly in response to oneor more visual signals, and may optionally prepare to take someappropriate action if the visual signal changes so as to indicate thatsuch an action is appropriate.

FIGS. 10-14 indicate meanings of different signals which may begenerated by the exemplary multicolor LED indicators described herein.FIG. 10 shows a potential lexicography for using the indicators as amanipulator user interface (“UI”), while FIG. 11 illustrates somegeneral meanings of signal colors from the indicators. FIG. 12 showsalternative general meanings of the two three-color LED indicators 20included in each manipulator assembly. By transmitting these or othercolor signals with sufficient brightness through a clear or translucentplastic cap at a position on the manipulator assembly along the toolinsertion axis away from the patient, the signals may be readilyidentified throughout a wide area. As the red, green, and blue elementsof each LED array have a brightness that can be independentlycontrolled, and as the two LED arrays of an indicator may beindependently controlled, a wide variety of identifiable signal colorsmay be generated.

FIG. 13 specifically identifies system, manipulator assembly, and/ortool states that may be associated with several specific signals fromthe manipulator assembly indicators. Synchronous blinking encompassesblinking of both LED arrays within a single indicator of a manipulatorassembly, as can be understood with reference to FIGS. 6 and 7.Alternate blinking encompasses energizing of the two LED arrays without-of-phase blinking, so that when one LED array is on, the other isoff. Still further additional signals may be generated by the indicatorsto communicate alternative information. For example, signals (such as aquick green blink) may indicate that a component (such as a cannula,sterile adapter, tool or surgical instrument, or the like) that hasrecently or just been mounted to the manipulator system is valid andaccepted. Alternative signals may indicate that a tool change isrequested by the system operator, that an energizable instrument (suchas an electrocautery instrument or the like) is being activated by thesystem operator, and the like. Different signal characteristics mayoptionally be associated with recoverable faults and unrecoverablefaults. Hence, a wide variety of still further alternative signals maybe included.

While described in some detail, for clarity of understanding and by wayof example, a variety of changes, adaptations, and modifications will beobvious to those of skill in the art. Hence, the scope of the presentinvention is limited solely by the appended claims.

What is claimed is:
 1. A medical system comprising: a master controller;a manipulator adapted to hold and manipulate each one of a firstplurality of medical devices, the first plurality of medical devicesincluding a first medical device mounted on the manipulator and aplurality of unmounted medical devices, the plurality of unmountedmedical devices including a replacement medical device; at least oneactivatable indicator corresponding to the manipulator; a plurality ofactivatable indicators, each of the plurality of activatable indicatorscorresponding to one of the plurality of unmounted medical devices; andat least one processor programmed to: control movement of themanipulator in response to manipulation of the master controller,control activation of the at least one activatable indicator so as tocommunicate information that the first medical device should be replacedby the replacement medical device, so that the replacement medicaldevice is mounted on the manipulator, and control activation of theplurality of activatable indicators so as to indicate the replacementmedical device by activating the activatable indicator corresponding tothe replacement medical device.
 2. The medical system of claim 1,further comprising: a structure upon which the plurality of unmountedmedical devices are each individually and removably placed, wherein eachof the plurality of activatable indicators is visually associated with acorresponding one of the plurality of unmounted medical devices.
 3. Themedical system of claim 1, further comprising: a second mastercontroller; a second manipulator adapted to hold and manipulate one ofthe plurality of unmounted medical devices; at least one secondactivatable indicator corresponding to the second manipulator; and amonitor; wherein the at least one processor is programmed to: causeselectable indicators of the manipulator, the second manipulator, andthe plurality of unmounted medical devices to be displayed on themonitor; receive a selection of one of the selectable indicators of themanipulator and the second manipulator, and in response, activate atleast one activatable indicator corresponding to the selection of theone of the manipulator and the second manipulator, and receive aselection of one of the selectable indicators of the plurality ofunmounted medical devices, and in response, activate the activatableindicator corresponding to the selection of the one of the plurality ofunmounted medical devices.
 4. A medical system comprising: a mastercontroller; a manipulator adapted to hold and manipulate a medicaldevice; at least one activatable indicator corresponding to themanipulator; a second master controller; a second manipulator adapted tohold and manipulate a second medical device; at least one secondactivatable indicator corresponding to the second manipulator; and atleast one processor programmed to: control movement of the manipulatorin response to manipulation of the master controller, control activationof the at least one activatable indicator so as to communicate anindication that the medical device being held by the manipulator shouldbe replaced by a replacement medical device, control movement of thesecond manipulator in response to manipulation of the second mastercontroller, and control activation of the at least one secondactivatable indicator so as to communicate an indication that the secondmedical device being held by the second manipulator should be replacedby a second replacement medical device.
 5. The medical system of claim4, wherein the at least one processor is programmed to activate the atleast one activatable indicator if a first replacement commandassociated with the master controller is received, and to activate theat least one second activatable indicator if a second replacementcommand associated with the second master controller is received.
 6. Amedical system comprising: a master controller; a manipulator adapted tohold and manipulate a medical device; at least one activatable indicatorcorresponding to the manipulator; a monitor; and at least one processorprogrammed to: control movement of the manipulator in response tomanipulation of the master controller, control activation of the atleast one activatable indicator so as to communicate information thatthe medical device being held by the manipulator should be replaced by areplacement medical device, receive information that identifies each oneof a plurality of unmounted medical devices, generate a list ofavailable medical devices by using the received information thatidentifies each one of the plurality of unmounted medical devices, anddisplay the list of available medical devices on the monitor.
 7. Themedical system of claim 6, further comprising: a plurality ofactivatable indicators, each of the plurality of activatable indicatorscorresponding to one of the plurality of unmounted medical devices;wherein the at least one processor is programmed to respond to aselection of the replacement medical device from the list of availablemedical devices by activating a corresponding one of the plurality ofactivatable indicators.